1. Field of the Invention
This invention relates to access and finding a position in video or other recording media. The invention is more particularly related referencing a position within recorded media using a reference marker related to the position. A note, portion of the media itself, or other item related to the position may be utilized as the reference-marker. The invention is still further related to the use of glyph encoding for associating the reference marker to the corresponding position in the recorded media.
2. Discussion of the Background
Multimedia note-taking systems typically capture audio and video during a meeting and slides are created from the captured material. (For example, Tivoli, a system running on LiveBoard, see Moran, T. P., Palen, L., Harrison, S., Chiu, P., Kimber, D., Minneman, S., van Melle, W., and Zellweger, P. “I'll get that off the audio”: a case study of salvaging multimedia meeting records. Proceedings of CHI '97 CM, New York, pp. 202-209). Tivoli is designed to support working meetings rather than presentation meetings. The ink strokes in Tivoli, which are indexed to the audio, along with any prepared material on the Tivoli slides become the group notes to the meeting. A participant using a laptop may “beam” typed text comments onto a slide in Tivoli.
In a similar example, Classroom 2000, images of presentation slides and audio are captured, but video is not used (see Abowd, G. D., Atkeson, C. G., Brotherton, J., Enqvist, T., Gulley, P., and LeMon, J. Investigating the capture, integration and access problem of ubiquitous computing in an educational setting, Proceedings of the CHI '98 Conference. ACM, New York, pp. 440-447; and Abowd, G. D., Atkeson, C. G., Feinstein, A., Hmelo, C., Kooper, R., Long, S., Sawhney, N., and Tani, M., Teaching and learning as multimedia authoring: the classroom 2000 project, Proceedings of the ACM Multimedia '96 Conference. ACM, New York, pp. 187-198). In addition, Classroom 2000 requires effort by the presenter to prepare the slides in a standard graphics format. The slides are displayed on a LiveBoard and note-taking is done with PDA devices pre-loaded with slides. These notes are later synchronized to the audio and the slides which have been annotated by the professor lecturing in front of the LiveBoard.
In yet another example, the Forum (see Isaacs, E. A., Morris, T., and Rodriguez, T. K. A forum for supporting interactive presentations to distributed audiences. Proceedings of CSCW '94. ACM, New York, pp. 405-416), is a system uses video as a means for distributed presentations. Everyone, including the speaker, sits in front of a workstation during a presentation. Slides have to be prepared in a specified format. The slides can be annotated with text and marks drawn with a mouse, but the video images cannot be annotated.
In another example, the STREAMS (see Cruz, G., and Hill, R. Capturing and playing multimedia events with STREAMS. Proceedings of the ACM Multimedia '94 Conference. ACM, New York, pp. 193-200), is a system for presentation capture that uses video from room cameras. These cameras are also used to capture any presentation content on display. This method has problems when activity in the room obscures the display. Note-taking during the presentation is not supported, although the captured video streams can be annotated during review by adding text comments. None of these systems allow interactive integration of live images from cameras and presentation material into the notes.
In addition, there are also several known stand alone ink and audio note-taking systems. For example, FXPAL Dynomite (see Wilcox, L. D., Schilit, B. N., and Sawhney, N. Dynomite: A Dynamically Organized Ink and Audio Notebook. Proceedings of CHI '97. ACM, New York, pp. 186-193); and Audio Notebook (see Stifelman, L. The Audio Notebook: Paper and Pen Interaction with Structured Speech. Ph.D. Thesis. MIT, 1997), which uses paper with audio recording. Filochat (see Whittaker, S., Hyland, P., and Wiley, M. Filochat: handwritten notes provide access to recorded conversations. Proceedings of CHI '94. ACM, New York, pp. 271-276), is a PC computer with a pen tablet in which audio is indexed with handwritten notes; and NoTime (see Lamming, M., and Newman, W. Activity-based information technology in support of personal memory. Technical Report EPC-1991-103, Rank Xerox, EuroPARC, 1991), was designed to key the user's ink strokes to recorded audio or video.
Also known are video annotation systems. Marquee (see Weber, K., and Poon, A. Marquee: a tool for real-time video logging. Proceedings of CHI '94. ACM, New York, pp. 58-64) is a pen-based system for making annotations while watching a videotape. A later version of Marquee has modifications to take timestamps on digital video streams from the WhereWereWe multimedia system (see Minneman, S., Harrison, S., Janssen, B., Kurtenbach, G., Moran, T., Smith, I., and van Melle, B. A confederation of tools for capturing and accessing collaborative activity. Proceedings of the ACM Multimedia '95 Conference. ACM, New York, pp. 523-534).
Vanna (see Harrison, B., Baecker, R. M. Designing video annotation and analysis systems, Graphics Interface '92. Morgan-Kaufmann, pp. 157-166); and EVA (see MacKay, W. E. EVA: An experimental video annotator for symbolic analysis of video data, SIGCHIBulletin, 21 (2), 68-71. 1989. ACM Press) are text based systems. VideoNoter (Trigg, R. Computer support for transcribing recorded activity, SIGCHI Bulletin, 21 (2), 68-71. 1989. ACM Press) displays and synchronizes different streams of activity (video, figures whiteboard drawings, text), but requires post-production to transcribe text from the audio or extract drawings from a whiteboard. These systems are limited by their design based on using videotapes rather than digital video. None of these systems allow interactive integration of video images into the notes. Sharp Zaurus (Zaurus Operation Manual. Sharp Corporation, 1996) is a commercial product, which is a PDA with a digital camera attached. Digital photos can be taken and linked to handwritten notes.
Furthermore, many systems include video playback mechanisms. FIG. 1 illustrates a conventional video playback device 100 that includes a video display area 102. A timeline 105 shows a current position of the video being displayed, and standard VCR controls, including play 115, Fast Rewind 120, Fast Forward 125, Step Back 130, Step Forward 135, etc., are provided that allow a user to navigate the video. However, difficulties arise in finding specific positions within a video, particularly in reference to notes or other items related to the video.